Population assessment of health system performance in 16 countries

Abstract Objective To demonstrate how the new internationally comparable instrument, the People’s Voice Survey, can be used to contribute the perspective of the population in assessing health system performance in countries of all levels of income. Methods We surveyed representative samples of populations in 16 low-, middle- and high-income countries on health-care utilization, experience and confidence during 2022–2023. We summarized and visualized data corresponding to the key domains of the World Health Organization universal health coverage framework for health system performance assessment. We examined correlation with per capita health spending by calculating Pearson coefficients, and within-country income-based inequities using the slope index of inequality. Findings In the domain of care effectiveness, we found major gaps in health screenings and endorsement of public primary care. Only one in three respondents reported very good user experience during health visits, with lower proportions in low-income countries. Access to health care was rated highest of all domains; however, only half of the populations felt secure that they could access and afford high-quality care if they became ill. Populations rated the quality of private health systems higher than that of public health systems in most countries. Only half of respondents felt involved in decision-making (less in high-income countries). Within countries, we found statistically significant pro-rich inequalities across many indicators. Conclusion Populations can provide vital information about the real-world function of health systems, complementing other system performance metrics. Population-wide surveys such as the People’s Voice Survey should become part of regular health system performance assessments.


Introduction
The notion that health systems should be people-centred seems unexceptional.Health systems are occupied with serving people, are funded by people and aim to improve people's health as their primary objective.2][3][4][5][6][7] The global drive towards universal health coverage (UHC) also relies on a social compact that presumes the population finds health services to be of high value.
][10] However, rhetoric on peoplecentredness has exceeded reality. 9The recently developed UHC framework for health system performance assessment (Fig. 1) highlights the need to evaluate health systems based on how they function for people and the outcomes they generate. 11Building on a body of literature that is arguing for a shift from measuring inputs to assessing health system function and health improvement, [12][13][14] this emphasis on performance is especially relevant today with many health systems struggling in the aftermath of the coronavirus disease 2019 (COVID- 19)  pandemic.Incorporating people's perspective in evaluating and steering health systems will require a robust and comparable set of measures obtained from the population. 15n this paper we describe the evaluation of health system performance by 16 different populations using the People's Voice Survey (data publicly available in mid-2024), 16 a new internationally comparable instrument. 17We hypothesize that the survey captures many of the domains in the WHO framework for health system performance assessment, and that it can provide unique insights into performance to complement other data sources.We report and discuss data from 16 countries on performance domains, and examine income-related inequality by analysing differences between income groups within each country.

Indicators
We based our survey indicators on the WHO UHC framework for health system performance assessment (Fig. 1), 11 which has core areas of service delivery, intermediate objectives and final goals.The domains within these core areas are broadly consistent with many widely used frameworks. 11,14,18,19We identified survey data that corresponded to the concepts in the framework, and were able to quantify all intermediate objectives (except for safety) and all final goals (Fig. 1).We define the indicators used in our analysis in Box 1, mapping them to the framework constructs.Where possible, to better capture the core construct and to reduce statistical noise, we combined multiple variables from the survey.
In the intermediate objectives area of the framework, care effectiveness refers to the ability of the overall health-care system to provide essential and clinically effective services to those who need them. 19User experience refers to the provision of care that is respectful and aligns with individual preferences, needs and values. 20Access is the availability and timely delivery of health-care services. 21n the final goals area of the framework, people-centredness encompasses the ability of systems to capture the public's input, perceptions of quality, choice of provider, engagement in care and trust in the system. 10,11Health improvement includes morbidity and mortality. 19For financial protection we included the weighted proportion of respondents with health insurance 22 as well as perceived health security, developed as a peoplereported measure of UHC. 23e also calculated domain score averages and plotted these against national health spending per capita; we used Excel (Microsoft, Redmond, USA) to calculate Pearson correlation coefficients (r).

Statistical analysis
We constructed post-stratification weights according to country-specific demographic variables to account for differences in sample design and probability of selection.Numbers of respondents and percentages presented are therefore weighted.We captured demographic data to allow an equity analysis (online repository). 24We performed all analyses using Stata version 15.0 (StataCorp, College Station, USA).We created the circumplex plots (coxcombs) using R (R Core Team, Vienna, Austria) and the scatter plots using Excel.
To assess income-related inequalities within countries, we calculated the slope index of inequality (online repository). 25The slope index expresses the absolute percentage point difference in health system outcome between the predicted poorest and wealthiest in the income distribution, assuming a linear relation between income rank and the outcome. 26We used logistic regression and estimated the marginal effects using the lincom post-estimation command in Stata.We used within-country income group categories (online repository) 27 to construct the equiplots.

Ethics
The QuEST hub at Harvard, Boston, USA, and collaborators in participating countries obtained ethical clearance for the People's Voice Survey as required by local regulations.As the survey presented minimum risk to participants, the Harvard Human Research Protection Program determined the research to be exempt from human subjects considerations.We provide details in the online repository. 28

Results
The number of participants in the 16 countries included in the survey totalled 27 795, ranging from 1001 (Italy) to 2779 (Ethiopia; Table 1 available at https:// www .who.int/publications/ journals/ bulletin).We provide the weighted survey results for intermediate objectives and final goals in both data format (Table 1) and in coxcomb plots for visual interpretation (Fig. 2 and Fig. 3).

Intermediate objectives
We observed that the weighted proportion of people aged 40 years or older, who reported having had their blood pressure and blood sugar checked within the past year (public health effectiveness) was an average of 43

Intermediate objectives
• Care effectiveness: (i) public health effectiveness: percentage of respondents aged ≥ 40 years who had both a blood pressure and blood sugar test in the past year; (ii) quality of own care: percentage of respondents rating quality of care of most recent visit in past 12 months as very good or excellent; and (iii) quality of primary care services: average percentage of respondents rating three core primary care services (child, maternal, chronic disease) as very good or excellent.• User experience: (i) respect: percentage of respondents rating respect that provider showed them and courtesy of office staff in most recent visit as very good or excellent, and who experienced no discrimination in health care; (ii) voice: percentage of respondents rating their desired level of involvement in their health care and their health-care provider's explanation as very good or excellent; and (iii) customer service: percentage of respondents rating wait time and time spent with provider (as well as time waiting for appointment in six countries with appointment systems) as very good or excellent.• Access: (i) connection to health system: percentage of respondents with usual source of care; (ii) use of needed health care: percentage of respondents with chronic disease who used care at least once in past year; and (iii) no unmet need: percentage of respondents with no unmet health care needs in past year.

Final goals
• People-centredness: (i) quality of public health system: percentage of respondents rating quality of the country's public health system as very good or excellent; (ii) quality of private health system: percentage of respondents rating quality of the country's private health system as very good or excellent; (iii) endorsement: percentage of respondents reporting that the health system works well as it is/needs only minor change; and (iv) involvement in decision-making: percentage of respondents rating that government considers public opinion as very good or excellent.• Health improvement: (i) self-rated health: percentage of respondents reporting their overall health as very good or excellent; (ii) self-rated mental health: percentage of respondents reporting their mental health as very good or excellent; and (iii) absence of disease: percentage of respondents who do not have a chronic/longstanding condition.In the domain of user experience, the indicators respect, voice and customer service demonstrated similar ratings between countries; approximately one-third of respondents rated their last health-care visit as very good or excellent for these items.The highest endorsement of voice was observed in the USA (65.6%; 983/1500), followed by Greece (59.0%; 1184/2010), the United Kingdom (53.5%; 897/1677) and Argentina (47.6%; 567/1190).We noted the highest ratings for customer service in Greece (60.7%; 1023/2010), followed by Nigeria (51.6%; 974/2555) and the USA (45.7%; 629/1500).
Measures for access received the highest endorsement of all the indicators in this study.The percentage of people with a usual source of care (i.e.connection to a health system) was 73.2% (20 265/27 795) across all countries, with the highest weighted proportion in Uruguay (93.8%; 1154/1237) followed by Lao People's Democratic Republic (88.5%; 1740/2007) and the United Kingdom (87.6%; 1448/1677).We observed that use of needed health care among people with chronic illness was reported as greater than 80% in all countries.The percentage of people reporting no unmet need was also very high across all countries, and ranged from the lowest in Peru (74.2%; 931/1255) to the highest in the Republic of Korea (94.0%; 1881/2000; Table 1 and Fig. 2).
With regards to people-centredness, the survey revealed that the quality of the public or government health system and the private health system was perceived as very good or excellent by only 26.2% 7208/27 795) and 38.5% (10 197/27 795) of respondents, respectively.We observed the highest percentage of people rating the government health system highly in the Republic of Korea (42.4%; 847/2000), closely followed by the United Kingdom (42.3%; 703/1677); in all other countries, around one third or less of respondents did not rate their country's health system as being of good quality.Agreement that the health system is working well as it is (i.e.endorsement) received the lowest scores of all indicators in this analysis; we note an average value for this indicator of only 24.6% (6758/27 795).Overall, an average of 52.1% (14 192  1 and Fig. 3).

Correlation with health spending
We calculated how several of the performance indicators were correlated with national health spending per capita.We noted the strongest correlation with health spending for user experience (r = 0.62), followed by care effectiveness (r = 0.57), financial protection (r = 0.51) and access (r = 0.37; Fig. 4).We observed no association between total health expenditure per capita and health improvement (r = −0.03),and a negative correlation between spending and people-centredness (r = −0.24).

Income-based inequity
Fig. 5 shows that people with the lowest incomes within any specific country were less positive across the majority of the intermediate objective indicators.The results of final goals by income group are available in the online repository. 29We observe the greatest pro-rich inequities within Italy, Kenya, Lao People's Democratic Republic, Mexico, Republic of Korea, South Africa and Uruguay in the domain of financial protection (online repository). 25ncome-related inequities were also substantial within countries, with all countries except India and Peru showing a pro-rich difference of 25% or more between the richest and poorest respondents for at least one of the indicators (see slope index of inequality data in online repository). 25Our data highlight pro-poor differences for India for connection to health system (access) by 27%, and for Peru for endorsement (people-centredness) by 28%.The largest pro-rich inequalities were found for self-rated physical and mental health, and insurance.For example, compared with the lowest-income respondents, the weighted proportion of highestincome people reporting health insurance in Mexico was 54% higher, and the weighted proportion of highest-income people rating their health as very good or excellent in the United Kingdom was 47% higher (online repository). 25

Discussion
Despite efforts to improve the accountability of health systems to populations, assessments of health system performance have not prioritized people's experiences and perspectives. 15any of the aspects of performance obtained in the People's Voice Survey are uniquely available from people, while other results provide complementary insights to other data sources.Data from the People's Voice Survey highlighted several positive aspects of health system performance, but also uncovered major deficits.
Care effectiveness is a key signal of health system function.For example, fewer than half of respondents across countries (age ≥ 40 years) had received a blood pressure and blood glucose check in the past year.Screening and regular monitoring of these parameters in older adults is critical

Research
Bull World Health Organ 2024 ; 102 : 486-497A| doi: http://dx.doi.org/10.2471/BLT.23.291184   for early initiation of prevention and control measures for cardiovascular disease and diabetes, now the leading causes of disease, death and disability in most low-and middle-income countries. 30,31No country achieved over 50% positive ratings in the three core primary care services (maternal, child, chronic disease care).3][34][35][36] The low primary care ratings and the large divergence between people's ratings of their own care and overall primary care may indicate a perception that good care is only available through the individual's own efforts. 37,38oor user experience can reduce care seeking and adherence, and undermine health outcomes and confidence. 39,40We found that respect, voice and customer service were rated positively by an average of only one in three respondents across countries studied.Ratings tended to be lower in lower income countries, and the correlation with national health spending was highest of all the domains.Historically, user experience has had more policy attention in wealthier countries. 41The findings here should provide motiva-tion to policy-makers in all countries to pursue solutions, including via medical education, management, supervision and more responsive user feedback.
The domain of access demonstrated the best performance of all the intermediate objectives, although there is still work to be done to achieve universal coverage.In most countries, at least three quarters of people reported no unmet need for health care; similarly high proportions of people with chronic disease had at least one health-care visit in the past year.We observed greater variation in respondents reporting a usual source of care.Having a usual source of care is a (inexact) proxy for primary care, and is associated with a higher uptake of preventive services and a positive experience of care. 42,435][46] The high levels of reported access in India, sub-Saharan African countries and parts of Latin America contrast with the substantial excess mortality from treatable conditions in these regions because of poor-quality care. 1,47n terms of people-centredness, feedback on the quality of public health systems was overall rather negative; only one in four respondents across the 16 countries rated their government health systems highly.In all countries, except Ethiopia and the Republic of Korea, private health systems outperformed public health systems.Health systems across the 16 countries are predominantly public, government-owned or based on social security.The exceptions are the Republic of Korea and the USA, where most health care is provided by the private sector.The gap between private and public health system approval serves as a measure of how far public services are lagging behind private services; if large, this gap can indicate a need for policy-makers to learn about what works in the private sector.On average, only half of respondents felt their governments considered their opinions when making health policies.Other studies found that users in both low-and highincome countries are dissatisfied with their health systems. 14,48ur observation that people-centredness measures are negatively correlated with health spending is a result of lower endorsement and poor government responsiveness to user feedback  in wealthier countries.This finding requires further study, but suggests that spending on health does not prevent populations from feeling alienated from their health systems.One implication is that policy-makers should more meaningfully involve the population in system reform, and should increase their efforts to better communicate the work and achievements of the health system to the population. 49he health improvement measures in the People's Voice Survey (self-rated health, self-rated mental health and no chronic disease) reflect social determinants of health and public health, as well as health systems.These indicators are therefore not as well-suited to measuring health system performance as other more specific health system metrics, such as amenable mortality. 1,2,50ith regards to financial protection, we observe that insurance does not buy health security.Although an average of two thirds of the respondents had health insurance, only half of the respondents felt they could access and afford good-quality care if very sick.This perceived poor health security could be considered a measure of the effectiveness of UHC. 23Although financial protection received higher ratings in wealthier countries, there were still major shortcomings.In the nine countries with universal health insurance (> 90% coverage), less than half of respondents felt health-secure.2][53] These findings have important implications for the design of health benefit packages, and for timely and effective delivery of insured services.
We observed that perceived and/ or experienced health system performance varied by respondents' incomes.Although access ratings were consistent between the various income groups within most countries, we found poorer ratings of care effectiveness among lower-income respondents in four countries.We noted the largest inequities in user experience, with around one half of the countries showing a pro-rich pattern.5][56] These data may underestimate inequities as poorer and less educated respondents may have lower expectations and positively rate low-quality care. 57ur study had several limitations.First, the People's Voice Survey was developed using the conceptual framework of the Lancet Global Health Commission on High Quality Health Systems and did not contain all indicators within the WHO framework (e.g.safety). 17Second, people's perceptions of health systems are influenced by various cultural, political, social and personal factors, including education level, as well as individual health and the effect of any health care received. 58Because the recent COVID-19 pandemic may have influenced people's responses, the survey should be repeated every 2 years to gain an understanding of performance and trajectory.Third, in some countries the prevailing low quality of health care may reduce people's expectations and therefore inflate ratings, complicating cross-country comparison.Comparisons are therefore most valid for countries with similar income levels and health systems. 57Fourth, the survey sampled the entire population and did not have sufficient resolution to reflect the experiences of potentially vulner-able subgroups (e.g.recent migrants, homeless or older populations), who may have very different perceptions and experiences of a health system; to begin to address this limitation, QuEST collaborators are currently pursuing studies focused on migrants and adolescents.
We found that the elements of the WHO framework were relevant and measurable (except for safety) from the survey responses.One area that could be further refined in the framework is people-centredness.Since this element is at the core of the health system, the concept might be integrated throughout the other dimensions of the framework (e.g.voice as part of user experience).Further, given the central role of trust in the health system, confidence (including health system endorsement and health security) could be added as an impact indicator of system performance. 23o conclude, we have shown that populations can provide rich and nuanced information about the function of their health systems.These data not only describe health system performance but also provide signals of strength and weakness to guide policy.We found that in all countries people are generally unsatisfied with their current health Fig. 5
Resultados En el ámbito de la eficacia de la atención, se encontraron importantes deficiencias en los exámenes de salud y en la aprobación de la atención primaria pública.Solo uno de cada tres encuestados declaró tener una experiencia de usuario muy buena durante las visitas sanitarias, con porcentajes más bajos en los países de ingresos bajos.El acceso a la atención sanitaria fue el mejor valorado de todos los ámbitos; sin embargo, solo la mitad de las poblaciones se sentían seguras de poder acceder y costearse una atención de alta calidad en caso de enfermar.En la mayoría de los países, la población valoró más la calidad de los sistemas sanitarios privados que la de los públicos.Solo la mitad de los Note: The frequencies for number of respondents are rounded up to the nearest whole number as weighting can produce decimal frequencies.The indicator includes questions only asked to participants who reported a visit to a health facility in the last 12 months.
The indicator shows the average percentage of 3 questions.People were included if they had rated at least one of the questions as very good or excellent responses on the Likert scale. e The indicator includes an additional question about the time spent waiting for an appointment that was only asked in six countries with appointment systems (Greece, Italy, Mexico, Republic of Korea, United Kingdom and USA).
The indicator is only for participants with chronic disease.

Fig. 1 .
Fig. 1.WHO framework used to define indicators in a study on the use of People's Voice Survey indicators in a 16-country assessment of health system performance

Fig. 3 .
Fig. 3. Survey responses to final goals indicators of health systems performance assessment: people-centredness, health improvement and financial protection, 2022-2023

Fig. 4 .
Fig. 4. Correlation between average performance on care effectiveness, user experience and people-centredness and national health spending per capita, 2022-2023

a
See Box 1 for definitions of survey indicators.b The indicator only includes participants aged 40 years or older.c

User experience Argentina Colombia Ethiopia Greece Italy India Kenya Lao People's Democratic Republic Mexico Nigeria Peru Republic of Korea South Africa United Kingdom USA Uruguay
Note: See Box 1 for definitions of survey indicators.Research Bull World Health Organ 2024 ; 102 : 486-497A| doi: http://dx.doi.org/10.2471/BLT.23.291184